Military aircraft and other vehicles are frequently equipped with head-mounted displays (HMDs) through which the operator of the vehicle can see objects in the environment. The HMD displays imagery, especially tactical or other symbology, superimposed over objects in the real world seen from the vehicle to which the information in the imagery relates.
While HMDs are often monocular, the industry is beginning to more frequently employ binocular HMDs. i.e., HMDs with two separate eye displays, usually implemented with a helmet visor that has a semi-reflective mirror over each eye of the operator. Each eye of the operator can see the real world outside the canopy or windshield of the vehicle, and can also see an HMD virtual image reflected in the associated semi-mirror, which is seen as superimposed over the view of the real world.
A diagram of operation of a real-world HMD is seen in FIG. 9. The HMD has two separate eye displays 101, 102 for the left and right eyes L and R. These eye displays 101 and 102 are semi-reflective mirrors, and the user's eyes L and R look through the respective HMD eye displays 101 and 102 and through the vehicle window or canopy 103 at a real object or group of objects that is rarely closer than hundreds of meters, which is, for all intents and purposes, at an infinite distance. Since the eyes are looking at an object at infinity, the lines of sight A and B are therefore parallel to each other, and there is no parallax effect.
FIG. 13 is a schematic diagram of a real HMD system. A head tracker 107 detects the position of the user's head and transmits a signal with data defining the location. From this data and other data of the simulation and the virtual environment, a two-dimensional imagery generator 109 generates a 2-D image, and this image is transmitted and displayed on the left and right eye displays 101 and 102 of the helmet worn by the user. The images displayed on eye displays 101 and 102 are identical to each other, and the images or symbology of the eye displays are in the same location on both of the eye displays. The HMD imagery for both eyes aligns with the distant object or objects to which it relates in the real world, because as shown in FIG. 9 there is no parallax change between the view from one eye relative to the other when the objects to which the imagery relates are at a very great, effectively infinite, distance.
FIG. 10 is a diagram of a simulator with a screen 104 equipped with a real-world binocular HMD with left and right eye displays 101 and 102. These eye displays 101 and 102 are semi-reflective minors as in FIG. 9, and the user can see through them and see the screen 105.
An out-the-window scene, including a virtual object 104 in the virtual simulation environment, is created by a computerized image generator and projected onto a screen 105 that is fairly close, e.g., 40 inches from the operator's eyes. The closeness of the projected images produces some perceptual issues. As is natural and automatic for human binocular vision, the lines of sight C and D triangulate and converge at a convergence angle α when the eyes are looking directly at the object 104 on screen 105, i.e., there is a vergence of the lines of sight C and D. Due to the distance between the eyes and the closeness of the screen 105 to the eyes, there is a parallax effect between the relative eye positions, and each eye sees the object 104 at a slightly displaced and possibly slightly altered perspective. The human brain has no difficulty processing this and sees the object 104 as a single image.
If a real-world HMD display were used with the two eye displays 101 and 102 showing identical HMD images to both eyes, the HMD images will not properly align with the object 104 of the OTW scenery for both eyes. FIGS. 11 and 12 illustrate the problem in somewhat exaggerated form as compared to a real-world HMD display.
In the left-eye view of FIG. 11 and the right-eye view of FIG. 12, the exemplary symbology or imagery 109 of the HMD eye displays 101 and 102 is shown as seen in front of the OTW imagery and the virtual object 104 as rendered on the screen 105, to which the symbology relates. The symbology and the object do not align in the same way in both displays.
The user's eyes will look at the object 104 and see it clearly with the rest of the OTW scene, but the symbology 109 will be seen as in two places relative to the object 104, resulting in a double image. This double-image effect is unrealistic and undesirable in a simulator, and presents a significant problem in training using a binocular HMD in combination with a projected OTW scene. The user may be able to shift back and forth between the OTW and the HMD images, but that creates eye strain or else increased visual cognition time for the user.
Normally, when a viewer simultaneously focuses on two different objects at varying distances, the human visual system is able to merge the two images created by each eve and present a single, merged image to a viewer. As the viewing distance changes, the human eye has the capability to change the optical power to maintain a clear focus by a process called accommodation, wherein the eyes converge or diverge the eyes depending on the distance of the object being viewed, and a viewer is typically unaware that it is happening. When viewing images on a screen at a nominal distance of 40 inches, however, if there is a difference of distance of one half to one inch over the field of view, it will produce a double-image effect in the user's perception, so that the user does not perceive the OTW scenery and the HMD imagery as fused.
To make things even more complicated, the screen may be planar but at a severe angle, depending on the position of the user's head, or the screen may be a spherical dome or a surface that is either curved complexly in three dimensions, or a faceted screen arrangement with planar screens angled relative to each other, as shown in U.S. published application no. 2009/0066858 A1 of James A. Turner published Mar. 12, 2009 and entitled DISPLAY SYSTEM FOR HIGH-DEFINITION PROJECTORS, which is herein incorporated by reference. The relative parallax effect of viewing the screen arrangement from one eye location relative to the other may be quite complicated as a result.
Moreover, the HMD imagery may be more complex than just character-based symbology, and may be a detailed field of view, such as a Forward Looking Infra-Red (FLIR) image. Use of a standard HMD imagery system in such a simulation would make the quality of the virtual training unrealistic and well below the quality of display found in the real vehicle, reducing the value of such training.
Systems have attempted to avoid these vergence problems by employing only a monocular HMD. However, the monocular HMD simulation is not the same experience as the binocular HMD real vehicle operation.
Systems have also provided a binocular HMD in simulation by maintaining the imagery of both eyes statically fixed for looking at displayed objects at a predetermined distance, e.g., the distance to a centerpoint on one of the screens of the screen arrangement. However, when the user starts to look around or move his or her head, the distance and relative angle of the screen varies, and there are variations in the vergence looking at the objects displayed on the screen, with the result that the HMD imagery is not binocularly convergent with the out-the-window imagery.
There are no systems in the prior art that provide for an adequate operation of a binocular HMD in a simulator where out-the-window imagery is displayed on screens or similar display apparatus.